Payload data handling, telemetry and data compression systems for Gaia.

Gaia, la nova missió astromètrica de la ESA amb un llançament previst pel 2011, observarà més de mil milions d'estels i altres objectes amb una exactitud sense precedents. Els seus ambiciosos objectius desbanquen completament les missions rivals d'altres agències. Al final de la seva vida útil es generarà el major i més complert mapa tridimensional de la nostra Galàxia.Una missió com aquesta suposa grans esforços tecnològics i de disseny ja que caldrà detectar, seleccionar i mesurar centenars d'estels cada segon, per enviar-ne posteriorment les dades cap a la Terra -a més d'un milió i mig de quilòmetres. Hem centrat el treball d'aquesta tesi en aquesta vessant de la missió, proposant dissenys pels sistemes de gestió de dades, de telemetria científica, i de compressió de dades. El nostre objectiu final és fer possible la transmissió a l'estació terrestre d'aquesta immensa quantitat de dades generades pels instruments, tenint en compte la limitada capacitat del canal de comunicacions. Això requereix el disseny d'un sistema de compressió de dades sense pèrdues que ofereixi les millors relacions de compressió i garanteixi la integritat de les dades transmeses. Tot plegat suposa un gran repte pels mètodes de la teoria de la informació i pel disseny de sistemes de compressió de dades.Aquests aspectes tecnològics encara estaven per estudiar o bé només es disposava d'esborranys preliminars -ja que la missió mateixa estava en una etapa preliminar en quan varem començar aquesta tesi. Per tant, el nostre treball ha estat rebut amb entusiasme per part de científics i enginyers del projecte.En primer lloc hem revisat l'entorn operacional del nostre estudi, descrit a la primera part de la tesi. Això inclou els diversos sistemes de referència i les convencions que hem proposat per tal d'unificar les mesures, referències a dades i dissenys. Aquesta proposta s'ha utilitzat com a referència inicial en la missió i actualment altres científics l'estan ampliant i millorant. També hem recopilat les principals característiques de l'instrument astromètric (en el qual hem centrat el nostre estudi) i revisat les seves directrius operacionals, la qual cosa també s'ha tingut en compte en altres equips.A la segona part de la tesi descrivim la nostra proposta pel sistema de gestió de dades de la càrrega útil de Gaia, la qual ha estat utilitzada per presentar els requeriments científics als equips industrials i representa en sí mateixa una opció d'implementació viable (tot i que simplificada). En la següent part estudiem la telemetria científica, recopilant els camps de dades a generar pels instruments i proposant un esquema optimitzat de codificació i transmissió, el qual redueix la ocupació del canal de comunicacions i està preparat per incloure un sistema optimitzat de compressió de dades. Aquest darrer serà descrit a la quarta i última part de la tesi, on veurem com la nostra proposta compleix gairebé totalment els requeriments de compressió, arribant a duplicar les relacions de compressió ofertes pels millors sistemes estàndard. El nostre disseny representa la millor solució actualment disponible per Gaia i el seu rendiment ha estat assumit com a disseny base per altres equips.Cal dir que els resultats del nostre treball van més enllà de la publicació d'una memòria de tesi, complementant-la amb aplicacions de software que hem desenvolupat per ajudar-nos a dissenyar, optimitzar i verificar la operació dels sistemes aquí proposats. També cal indicar que la complexitat del nostre treball ha estat augmentada degut a la necessitat d'actualitzar-lo contínuament als canvis que la missió ha sofert en el seu disseny durant els cinc anys del doctorat. Per acabar, podem dir que estem satisfets amb els resultats del nostre treball, ja que la majoria han estat (o estan essent) tinguts en compte per molts equips involucrats en la missió i per la mateixa Agència Espacial Europea en el disseny final

Optimization of time data codification and transmission schemes: Application to Gaia

Gaia is an ambitious space observatory devoted to obtain the largest and most precise astrometric catalogue of astronomical objects from our Galaxy and beyond. On-board processing and transmission of the huge amount of data generated by the instruments is one of its several technological challenges. The measurement time tags are critical for the scientific results of the mission, so they must be measured and transmitted with the highest precision - leading to an important telemetry channel occupation. In this paper we present the optimisation of time data, which has resulted in a useful software tool. We also present how time data is adapted to the Packet Telemetry standard. The several communication layers are illustrated and a method for coding and transmitting the relevant data is described as well. Although our work is focused on Gaia, the timing scheme and the corresponding tools can be applied to any other instrument or mission with similar operational principles

High-Performance Lossless Compression of Hyperspectral Remote Sensing Scenes Based on Spectral Decorrelation

The capacity of the downlink channel is a major bottleneck for applications based on remotesensing hyperspectral imagery (HSI). Data compression is an essential tool to maximize the amountof HSI scenes that can be retrieved on the ground. At the same time, energy and hardware constraintsof spaceborne devices impose limitations on the complexity of practical compression algorithms.To avoid any distortion in the analysis of the HSI data, only lossless compression is considered in thisstudy. This work aims at finding the most advantageous compression-complexity trade-off withinthe state of the art in HSI compression. To do so, a novel comparison of the most competitive spectraldecorrelation approaches combined with the best performing low-complexity compressors of thestate is presented. Compression performance and execution time results are obtained for a set of47 HSI scenes produced by 14 different sensors in real remote sensing missions. Assuming onlya limited amount of energy is available, obtained data suggest that the FAPEC algorithm yields thebest trade-off. When compared to the CCSDS 123.0-B-2 standard, FAPEC is 5.0 times faster andits compressed data rates are on average within 16% of the CCSDS standard. In scenarios whereenergy constraints can be relaxed, CCSDS 123.0-B-2 yields the best average compression results of allevaluated methods

Image and radio-frequency data compression for OPS-SAT using FAPEC

OPS-SAT is an ESA technology demonstration cubesat which includes a colour camera, a Software Defined Radio (SDR) receiver and a powerful ARM processor. One of the experiments executed there is FAPEC, a high-performance and versatile data compression software. Among others, it features image compression and linear prediction coding algorithms, suitable for multi-band and baseband radio-frequency (RF) samples, respectively. Since its deployment on-board OPS-SAT in late 2020, FAPEC has allowed for downloading a large set of Earth Observation images. Recently, thanks to ESA Open Space Innovation Platform funds, these two algorithms from FAPEC are being improved to get better compression ratios and speeds, add video compression capabilities, and higher quality levels in case of lossy compression. A smart lossy approach is being developed for radio-frequency data, identifying the time segments with signal presence and quantizing noise-only samples to further reduce the size. In order to identify the really useful files to be downloaded from the satellite, on-board data analysis capabilities are being developed as well. In this work we present in-orbit results, recent developments and preliminary results obtained with the new algorithms on real data.Peer ReviewedPostprint (published version

Context-aware lossless and lossy compression of radio frequency signals

We propose an algorithm based on linear prediction that can perform both the lossless and near-lossless compression of RF signals. The proposed algorithm is coupled with two signal detection methods to determine the presence of relevant signals and apply varying levels of loss as needed. The first method uses spectrum sensing techniques, while the second one takes advantage of the error computed in each iteration of the Levinson–Durbin algorithm. These algorithms have been integrated as a new pre-processing stage into FAPEC, a data compressor first designed for space missions. We test the lossless algorithm using two different datasets. The first one was obtained from OPS-SAT, an ESA CubeSat, while the second one was obtained using a SDRplay RSPdx in Barcelona, Spain. The results show that our approach achieves compression ratios that are 23% better than gzip (on average) and very similar to those of FLAC, but at higher speeds. We also assess the performance of our signal detectors using the second dataset. We show that high ratios can be achieved thanks to the lossy compression of the segments without any relevant signal.This work was (partially) funded by the European Space Agency (ESA) Contract No. 4000137290, the Spanish Ministry of Science and Innovation projects PID2019-105717RB-C22 (RODIN) and PID2021-122842OB-C21, the ERDF (a way of making Europe) by the European Union, the Institute of Cosmos Sciences University of Barcelona (ICCUB, Unidad de Excelencia María de Maeztu) through grant CEX2019-000918-M, grant 2021 SGR 1033 by Generalitat de Catalunya (AGAUR), and fellowship FPI-UPC 2022 by Universitat Politècnica de Catalunya and Banc de Santander.Postprint (published version

High-Performance Compression of Multibeam Echosounders Water Column Data

Over the last few decades, multibeam echosounders (MBES) have become the dominant technique to efficiently and accurately map the seafloor. They now allow to collect water column acoustic images along with the bathymetry, which is providing a wealth of new possibilities in oceans exploration. However, water column imagery generates vast amounts of data that poses obvious logistic, economic, and technical challenges. Surprisingly, very few studies have addressed this problem by providing efficient lossless or lossy data compression solutions. Currently, the available options are only lossless, providing low compression ratios at low speeds. In this paper, we adapt a data compression algorithm, the Fully Adaptive Prediction Error Coder (FAPEC), which was created to offer outstanding performance under the strong requirements of space data transmission. We have added to this entropy coder a specific pre-processing stage tailored to theKongsbergMaritime water column file formats. Here, we test it on data acquired with Kongsberg MBES models EM302, EM710, andEM2040.With this bespoke pre-processing, FAPEC provides good lossless compression ratios at high speeds, whereas lossy ratios reach water column file sizes even smaller than bathymetry raw files still with good image quality. We show the advantages over other lossless compression solutions, both in terms of compression ratios and speed.We illustrate the quality of water column images after lossy FAPEC compression, as well as its resilience to datagram errors and its potential for automatic detection of water column targets. We also show the successful integration in ARM microprocessors (like those used by smartphones and also by autonomous underwater vehicles), which provides a real-time solution for MBES water column data compression

Gaia Data Release 2 - Photometric content and validation

Aims. We describe the photometric content of the second data release of the Gaia project (Gaia DR2) and its validation along with the quality of the data. Methods. The validation was mainly carried out using an internal analysis of the photometry. External comparisons were also made, but were limited by the precision and systematics that may be present in the external catalogues used. Results. In addition to the photometric quality assessment, we present the best estimates of the three photometric passbands. Various colour-colour transformations are also derived to enable the users to convert between the Gaia and commonly used passbands. Conclusions. The internal analysis of the data shows that the photometric calibrations can reach a precision as low as 2 mmag on individual CCD measurements. Other tests show that systematic effects are present in the data at the 10 mmag level

Gaia Early Data Release 3: Summary of the contents and survey properties

Context. We present the early installment of the third Gaia data release, Gaia EDR3, consisting of astrometry and photometry for 1.8 billion sources brighter than magnitude 21, complemented with the list of radial velocities from Gaia DR2. Aims. A summary of the contents of Gaia EDR3 is presented, accompanied by a discussion on the differences with respect to Gaia DR2 and an overview of the main limitations which are present in the survey. Recommendations are made on the responsible use of Gaia EDR3 results. Methods. The raw data collected with the Gaia instruments during the first 34 months of the mission have been processed by the Gaia Data Processing and Analysis Consortium and turned into this early third data release, which represents a major advance with respect to Gaia DR2 in terms of astrometric and photometric precision, accuracy, and homogeneity. Results. Gaia EDR3 contains celestial positions and the apparent brightness in G for approximately 1.8 billion sources. For 1.5 billion of those sources, parallaxes, proper motions, and the (GBP − GRP) colour are also available. The passbands for G, GBP, and GRP are provided as part of the release. For ease of use, the 7 million radial velocities from Gaia DR2 are included in this release, after the removal of a small number of spurious values. New radial velocities will appear as part of Gaia DR3. Finally, Gaia EDR3 represents an updated materialisation of the celestial reference frame (CRF) in the optical, the Gaia-CRF3, which is based solely on extragalactic sources. The creation of the source list for Gaia EDR3 includes enhancements that make it more robust with respect to high proper motion stars, and the disturbing effects of spurious and partially resolved sources. The source list is largely the same as that for Gaia DR2, but it does feature new sources and there are some notable changes. The source list will not change for Gaia DR3. Conclusions. Gaia EDR3 represents a significant advance over Gaia DR2, with parallax precisions increased by 30 per cent, proper motion precisions increased by a factor of 2, and the systematic errors in the astrometry suppressed by 30-40% for the parallaxes and by a factor ~2.5 for the proper motions. The photometry also features increased precision, but above all much better homogeneity across colour, magnitude, and celestial position. A single passband for G, GBP, and GRP is valid over the entire magnitude and colour range, with no systematics above the 1% leve

Gaia Data Release 1. Astrometry: one billion positions, two million proper motions and parallaxes

Context. Gaia Data Release 1 (DR1) contains astrometric results for more than 1 billion stars brighter than magnitude 20.7 based on observations collected by the Gaia satellite during the first 14 months of its operational phase. Aims. We give a brief overview of the astrometric content of the data release and of the model assumptions, data processing, and validation of the results. Methods. For stars in common with the Hipparcos and Tycho-2 catalogues, complete astrometric single-star solutions are obtained by incorporating positional information from the earlier catalogues. For other stars only their positions are obtained, essentially by neglecting their proper motions and parallaxes. The results are validated by an analysis of the residuals, through special validation runs, and by comparison with external data. Results. For about two million of the brighter stars (down to magnitude ∼11.5) we obtain positions, parallaxes, and proper motions to Hipparcos-type precision or better. For these stars, systematic errors depending for example on position and colour are at a level of ±0.3 milliarcsecond (mas). For the remaining stars we obtain positions at epoch J2015.0 accurate to ∼10 mas. Positions and proper motions are given in a reference frame that is aligned with the International Celestial Reference Frame (ICRF) to better than 0.1 mas at epoch J2015.0, and non-rotating with respect to ICRF to within 0.03 mas yr−1 . The Hipparcos reference frame is found to rotate with respect to the Gaia DR1 frame at a rate of 0.24 mas yr−1 . Conclusions. Based on less than a quarter of the nominal mission length and on very provisional and incomplete calibrations, the quality and completeness of the astrometric data in Gaia DR1 are far from what is expected for the final mission products. The present results nevertheless represent a huge improvement in the available fundamental stellar data and practical definition of the optical reference frame